Imaging device for associating image data with shooting condition information

ABSTRACT

An imaging device includes an imaging unit configured to generate image data, an image data analyzing unit configured to analyze the image data to determine an age group or a sex of an image of a person included in the image data, a voice data generating unit configured to generate voice data, a voice data analyzing unit configured to analyze the voice data, a shooting condition information generating unit configured to generate shooting condition information based on a result of an analysis by the voice data analyzing unit and the age group or the sex of the image of the person determined by the image data analyzing unit, an image data recording unit, and a recording controller configured to record the image data and the shooting condition information in the image data recording unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/555,272, filed on Nov. 26, 2014, based upon and claims the benefit ofpriority from Japanese Patent Application No. 2013-245218, filed on Nov.27, 2013, the entire contents of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging device that captures asubject.

2. Description of the Related Art

In an imaging device such as a digital camera, a capacity of an imagememory that records image data generated by imaging has recently beenincreasing.

The imaging device has a great advantage of providing an environment inwhich a user can capture with no concern for a memory capacity, sincethe number of images that can be recorded increases as described above.However, the user has to process enormous image data during playback.Specifically, some measures are required for a method of searching adesired image out of a huge number of images.

In view of this, a method has been proposed in which a search keywordinputted by a user is applied to image data upon recording this imagedata, and a desired image is searched based on this keyword (forexample, see Japanese Unexamined Patent Publication No. 2005-354134).

SUMMARY OF THE INVENTION

In accordance with some embodiments, an imaging device is presented.

In some embodiments, an imaging device includes: an imaging unitconfigured to capture an image of a subject and generate image data; animage data analyzing unit configured to analyze the image data generatedby the imaging unit to determine an age group or a sex of an image of aperson included in the image data, a voice data generating unitconfigured to generate voice data based on an input voice; a voice dataanalyzing unit configured to analyze the voice data generated by thevoice data generating unit; a shooting condition information generatingunit configured to generate shooting condition information related to ashooting condition, based on a result of an analysis by the voice dataanalyzing unit and the age group or the sex of the image of the persondetermined by the image data analyzing unit; an image data recordingunit configured to record therein the image data generated by theimaging unit; and a recording controller configured to associate theimage data generated by the imaging unit with the shooting conditioninformation generated by the shooting condition information generatingunit and record the image data and the shooting condition information inthe image data recording unit.

The above and other features, advantages and technical and industrialsignificance of this invention will be better understood by reading thefollowing detailed description of presently preferred embodiments of theinvention, when considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imagingdevice according to a first embodiment of the present invention;

FIG. 2 is a diagram illustrating one example of related informationrecorded in a related information recording unit illustrated in FIG. 1;

FIG. 3 is a flowchart illustrating an operation of the imaging deviceillustrated in FIG. 1;

FIG. 4 is a view illustrating one example of a shooting condition;

FIG. 5 is a view illustrating one example of an image corresponding toan image file generated by imaging under the shooting conditionillustrated in FIG. 4;

FIG. 6 is a block diagram illustrating a configuration of an imagingdevice according to a second embodiment of the present invention;

FIG. 7 is a flowchart illustrating an operation of the imaging deviceillustrated in FIG. 6;

FIG. 8 is a view illustrating one example of an image corresponding toan image file generated by the operation of the imaging deviceillustrated in FIG. 7;

FIG. 9 is a block diagram illustrating a configuration of an imagingdevice according to a third embodiment of the present invention;

FIG. 10 is a perspective view of the imaging device in FIG. 9 viewedfrom a photographer;

FIG. 11 is a perspective view of the imaging device in FIG. 9 viewedfrom a subject;

FIG. 12 is a diagram for describing related information recorded in therelated information recording unit illustrated in FIG. 9;

FIG. 13A is a diagram illustrating one example of related informationrecorded in the related information recording unit illustrated in FIG.9;

FIG. 13B is a diagram illustrating one example of related informationrecorded in the related information recording unit illustrated in FIG.9;

FIG. 14 is a view illustrating one example of an image corresponding toan image file generated by the operation of the imaging deviceillustrated in FIG. 9;

FIG. 15 is a block diagram illustrating a configuration of an imagingdevice according to a fourth embodiment of the present invention;

FIG. 16 is a diagram for describing related information recorded in therelated information recording unit illustrated in FIG. 15;

FIG. 17A is a diagram illustrating one example of related informationrecorded in the related information recording unit illustrated in FIG.15;

FIG. 17B is a diagram illustrating one example of related informationrecorded in the related information recording unit illustrated in FIG.15;

FIG. 18 is a flowchart illustrating the operation of the imaging deviceillustrated in FIG. 15;

FIG. 19A is a view illustrating one example of an image corresponding toan image file generated by the operation of the imaging deviceillustrated in FIG. 18;

FIG. 19B is a view illustrating one example of an image corresponding toan image file generated by the operation of the imaging deviceillustrated in FIG. 18;

FIG. 19C is a view illustrating one example of an image corresponding toan image file generated by the operation of the imaging deviceillustrated in FIG. 18;

FIG. 20 is a diagram illustrating one example of related informationaccording to a modification of the fourth embodiment of the presentinvention; and

FIG. 21 is a block diagram illustrating a configuration of an imagingsystem according to a fifth embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments for embodying the present invention (hereinafter referred toas embodiments) will be described below with reference to the drawings.The present invention is not limited to the embodiments described below.The same components are identified by the same reference numerals in thedescription of the drawings.

First Embodiment Schematic Configuration of Imaging Device

FIG. 1 is a block diagram illustrating a configuration of an imagingdevice 1 according to a first embodiment of the present invention.

The imaging device 1 is a device that captures a subject, and is anelectronic device such as a digital camera, a digital video camera, or acellular phone or a tablet portable device having a shooting function.The imaging device 1 according to the first embodiment captures asubject, and records image data generated by the imaging in associationwith shooting condition information related to a shooting condition uponthe imaging.

An essential part of the present invention will mainly be describedbelow as the configuration of the imaging device 1.

As illustrated in FIG. 1, the imaging device 1 includes an imaging unit11, a display unit 12, an input unit 13, a voice data generating unit14, a memory unit 15, a recording unit 16, and a control unit 17.

The imaging unit 11 captures a subject to generate image data under thecontrol of the control unit 17. The imaging unit 11 includes, forexample, an optical system that forms an image of the subject, an imagesensor, such as a CCD (Charge Coupled Device), that receives the subjectimage formed by the optical system and converts the received image intoan electric signal, and a signal processing unit that performs a signalprocess (A/D conversion) to the electric signal (analog signal) from theimage sensor to generate digital image data.

The image data generated by the imaging unit 11 is sequentially storedin the memory unit 15 under the control of the control unit 17, and isrecorded in the recording unit 16 according to a shooting operation tothe input unit 13 by a user (photographer) of the imaging device 1.

The display unit 12 is formed by using a display panel composed ofliquid crystal or organic EL (Electro Luminescence), for example. Thedisplay unit 12 displays a variety of images under the control of thecontrol unit 17. The display unit 12 also displays operation informationof the imaging device 1 and information related to shooting, asnecessary, under the control of the control unit 17.

The input unit 13 includes buttons and switches receiving a user'soperation, and outputs an instruction signal according to the user'soperation to the control unit 17.

The input unit 13 has a function as an operation receiving unitaccording to the present invention.

The voice data generating unit 14 generates voice data based on aninputted voice.

As illustrated in FIG. 1, the voice data generating unit 14 includes avoice input unit 141 and a voice processing unit 142.

The voice input unit 141 includes a microphone, and converts an inputtedvoice into an electric signal. The microphone (voice input unit 141) maybe disposed at any position such as a top surface of the imaging device1, a surface (front surface), facing a subject, of the imaging device 1,or a surface (back surface), facing a photographer, of the imagingdevice 1.

The voice processing unit 142 performs A/D conversion to the electricsignal converted by the voice input unit 141 with sampling andquantization to generate voice data under the control of the controlunit 17.

The voice data generated by the voice data generating unit 14 is storedin the memory unit 15 under the control of the control unit 17.

The memory unit 15 stores image data generated by the imaging unit 11 orvoice data generated by the voice data generating unit 14.

Specifically, the memory unit 15 has a function as a voice data storageunit according to the present invention.

As illustrated in FIG. 1, the recording unit 16 includes a programrecording unit 161, an image data recording unit 162, and a relatedinformation recording unit 163.

The program recording unit 161 records various programs (including animaging program) executed by the control unit 17, and various datapieces that are used during the execution of the programs.

The image data recording unit 162 records an image file in whichshooting condition information is associated with image data generatedby the imaging unit 11, under the control of the control unit 17.

FIG. 2 is a diagram illustrating one example of related informationrecorded in the related information recording unit 163.

The related information recording unit 163 records related informationin which a specific voice (extracted keyword) and shooting conditioninformation are associated with each other.

For example, as illustrated in FIG. 2, a “family photo” that is shootingcondition information is associated with “Dad” or “Mom” that is aspecific voice. A “school event” that is shooting condition informationis associated with a “teacher” that is a specific voice.

The control unit 17 includes a CPU (Central Processing Unit), forexample, for entirely controlling the operation of the imaging device 1with an instruction or data transfer to each unit in the imaging device1 according to an instruction signal from the input unit 13.

As illustrated in FIG. 1, the control unit 17 includes an imagingcontroller 171, a display controller 172, a voice data analyzing unit173, a shooting condition information generating unit 174, and arecording controller 175.

The imaging controller 171 allows the imaging unit 11 to capture asubject according to a photographer's shooting operation to the inputunit 13, and stores image data generated with the imaging by the imagingunit 11 into the memory unit 15 as recording image data that is to berecorded to the image data recording unit 162.

The display controller 172 controls the operation of the display unit 12to display an image on the display unit 12.

Specifically, the display controller 172 sequentially reads the latestimage data stored in the memory unit 15, and displays (live viewdisplay) the image corresponding to the image data onto the display unit12. The display controller 172 also reads an image file matching orsimilar to an inputted keyword, out of image files recorded in the imagedata recording unit 162, according to a playback operation (including anoperation of inputting a keyword upon searching an image) of “displayingan image corresponding to the recorded image data” to the input unit 13by the user of the imaging device 1, and displays the imagecorresponding to this image file on the display unit 12.

The voice data analyzing unit 173 reads voice data from the memory unit15, and analyzes the voice data while referring to related informationrecorded in the related information recording unit 163.

The shooting condition information generating unit 174 generatesshooting condition information by referring to related informationrecorded in the related information recording unit 163 based on theanalysis result by the voice data analyzing unit 173.

The recording controller 175 generates an image file in which theshooting condition information generated by the shooting conditioninformation generating unit 174 is associated with the recording imagedata stored in the memory unit 15, and records this image file on theimage data recording unit 162.

Operation of Imaging Device

Next, an operation of the imaging device 1 described above will bedescribed.

FIG. 3 is a flowchart illustrating the operation of the imaging device1.

When a power source of the imaging device 1 is turned on by thephotographer's operation to the input unit 13 (step S101: Yes), thecontrol unit 17 causes the imaging unit 11 to start a capture, andcauses the voice processing unit 142 to operate to start an acquisitionof voice (generation of voice data) (step S102: capturing step, voicedata generating step). The image data generated by the imaging unit 11is then sequentially stored in the memory unit 15.

Then, the display controller 172 reads the latest image data stored inthe memory unit 15, and causes the display unit 12 to start a live viewdisplay (step S103).

Subsequently, the imaging controller 171 determines whether a shootingoperation to the input unit 13 by the photographer is inputted or not(step S104).

When the imaging controller 171 determines that the shooting operationis not inputted (step S104: No), the imaging device 1 is returned tostep S101.

On the other hand, when determining that the shooting operation isinputted (step S104: Yes), the imaging controller 171 causes the imagingunit 11 to capture a subject, and stores (temporarily records) imagedata generated by the imaging with the imaging unit 11 into the memoryunit 15 as recording image data (step S105). With this, the recordingimage data stored in the memory unit 15 is updated to the latestrecording image data.

Next, the control unit 17 always monitors whether a certain period oftime has elapsed or not after the shooting operation is inputted (stepS106). When a certain period of time has elapsed (step S106: Yes), thecontrol unit 17 stops the operation of the voice processing unit 142(stops the acquisition of voice) (step S107).

The voice data generated in steps S102 to S107 is stored in the memoryunit 15. With this, the voice data stored in the memory unit 15 isupdated to the latest voice data.

Subsequently, the voice data analyzing unit 173 reads the voice datastored in the memory unit 15, and analyzes the voice data (step S108:voice data analyzing step).

Specifically, the voice data analyzing unit 173 determines whether ornot the voice data includes a specific voice forming the relatedinformation recorded in the related information recording unit 163during a predetermined period including a point at which the shootingoperation is executed (a point before this point by a predetermined timeto a point after this point by a predetermined time).

Then, the shooting condition information generating unit 174 generatesshooting condition information by referring to the related informationrecorded in the related information recording unit 163 based on theanalysis result by the voice data analyzing unit 173 in step S108 (stepS109: shooting condition information generating step).

For example, when determining that a specific voice forming the relatedinformation is included in the voice data as a result of the analysis bythe voice data analyzing unit 173, the shooting condition informationgenerating unit 174 reads (generates) shooting condition informationassociated with the specific voice from the related information as theshooting condition information that is to be associated with therecording image data.

On the other hand, when determining that a specific voice forming therelated information is not included in the voice data as a result of theanalysis by the voice data analyzing unit 173, the shooting conditioninformation generating unit 174 generates shooting condition informationindicating that a shooting condition is unclear.

Then, the recording controller 175 reads the recording image data storedin the memory unit 15 in step S105, generates an image file in which theshooting condition information generated in step S109 is associated withthe recording image data, and records this image file on the image datarecording unit 162 (step S110: recording control step).

Specific Examples of Shooting Condition and Image File

In order to describe in more detail the operation of the imaging device1 described above, specific examples of a shooting condition and animage file generated by capturing a subject under the shooting conditionwill be described.

FIG. 4 is a view illustrating one example of a shooting condition. FIG.5 is a view illustrating one example of an image corresponding to animage file generated by the imaging under the shooting conditionillustrated in FIG. 4.

It is supposed as a shooting condition that a mother Sm captures herdaughter Sd as a subject with an elephant Se in the background in a zooas illustrated in FIG. 4.

In this case, the mother Sm says to her daughter Sd, “Look this way”,before starting the shooting operation to the imaging device 1 (stepsS102 to S107). The daughter Sd says to her mother Sm, “Mom, elephant”,during a predetermined period including a point at which the mother Smperforms the shooting operation with the imaging device 1 (steps S102 toS107).

The imaging device 1 collects the word of the daughter Sd “Mom,elephant” and the word of the mother Sm “look this way” during stepsS102 to S107, and stores these words into the memory unit 15 as voicedata. The imaging device 1 also analyzes the voice data to determinethat the specific voice “Mom” forming the related information isincluded in the voice data (step S108). The imaging device 1 also readsthe “family photo” associated with the specific voice “Mom” from therelated information (FIG. 2) recorded in the related informationrecording unit 163 as the shooting condition information (step S109).The imaging device 1 generates an image file in which the “family photo”that is the shooting condition information is associated with therecording image data stored in the memory unit 15 in step S105, andrecords this image file in the image data recording unit 162 (stepS110).

After the image file is recorded as described above, the imaging device1 reads an image file, with which shooting condition informationmatching or similar to the inputted keyword (e.g., “family photo”), outof image files recorded in the image data recording unit 162, accordingto a playback operation to the input unit 13 by a user of the imagingdevice 1, and displays an image 1100 (FIG. 5) corresponding to thisimage file.

Specifically, as illustrated in FIG. 5, the image 1100 is an image inwhich character information T1 of “family photo” that is the shootingcondition information is superimposed on an image W1 corresponding tothe recording image data generated according to the shooting operationto the input unit 13.

The imaging device 1 described above according to the first embodimentanalyzes voice data generated based on an inputted voice to generateshooting condition information related to a shooting condition based onthe result of the analysis. The imaging device 1 then generates an imagefile in which image data generated by the imaging and the generatedshooting condition information are associated with each other, andrecords this image file on the image data recording unit 162.

With this configuration, the user of the imaging device 1 can search adesired image out of image files recorded on the image data recordingunit 162 by using the shooting condition information as a keyword. Sincethe shooting condition information serving as the keyword is generatedbased on a voice from a photographer or a subject, the user of theimaging device 1 has no need to input the shooting condition informationto the imaging device 1 every time he/she performs a shooting operation.

Accordingly, the imaging device 1 according to the first embodimentprovides an effect of enhancing search performance and a value uponviewing of an image without deteriorating usability.

The imaging device 1 according to the first embodiment refers to relatedinformation in which a specific voice and shooting condition informationare associated with each other upon analyzing voice data and generatingshooting condition information.

Therefore, upon the analysis of voice data, the imaging device 1 onlydetermines whether or not a specific voice forming the relatedinformation is included in the voice data, and upon the generation ofshooting condition information, the imaging device 1 only reads theshooting condition information associated with the specific voice in thevoice data. Accordingly, a processing load of the imaging device 1 canbe reduced.

The imaging device 1 according to the first embodiment also determineswhether or not a specific voice is included in voice data during apredetermined period including a point at which a shooting operation isperformed by a photographer.

Therefore, the imaging device 1 can generate shooting conditioninformation matching the actual shooting condition through an analysisof voice data during a period necessary for the generation of shootingcondition information. In addition, the imaging device 1 does not haveto analyze the entire voice data, thereby being capable of reducing aprocessing load upon analyzing the voice data.

Modification of First Embodiment

The above first embodiment describes a parent-child relationship as arelationship between a photographer and a subject (FIGS. 2 and 4).However, other relationship, e.g., a male-female relationship, can beemployed. In addition, this relation is not limited to popular humanrelationships (generally majority human relationships) described above,and this relation may be changed as necessary in consideration of arecent situation in which same-sex couples are present. A keyworddetermined from a voice may be based on a user's language or a dialector custom in a district. For example, it is obvious that an imagingdevice 1 is optimized by specifying an image of a user (photographer orsubject) and utilizing analysis data of big data. The first embodimentincludes an application in which an image is specified as a pet photo(shooting condition information) from a voice of a dog or a cat.Recently, a wearable camera has been developed. To create meaning to acaptured image by effectively utilizing a voice that can be detectedwherever the wearable camera is set can be expected to provide asignificant effect to specify a person or a pet with a wearable camera,since image data is difficult to be acquired in this situation.

In the above first embodiment, it is obvious that an application ofincluding feeling information concerning a feeling upon shooting intothe shooting condition information (recording feeling information inassociation with recording image data) by using a relaxed atmospherespecific to a family photo (voice has a moderate tempo, and has afeature such as a rise in pitch at the end) or a buzz specific to aschool event (various excited voices are detected, a hitting voice witha desk or a chair is heard, or heavy footsteps are heard) can beimplemented. An excited voice can be determined from features of manyplosive voices, being single, and including only a word, or a keywordthat is issued only under an excited situation. An analysis of feelingis possible from a simple word only by analyzing its intonation orrhythm.

The determination of even a feeling as described above increases anapplication range of the present invention, with the result that afeeling of a photographer or a subject as well as an objective conditionupon shooting can be reproduced. A viewer of an image not only simplyunderstands the image but also feels empathy for the image withemotional involvement. This is a significant motivation for viewing aphotograph, and the meaning of the present invention becomes clearer bythe assumption of such application.

Second Embodiment

Next, a second embodiment of the present invention will be described.

In the description below, the configurations and steps same as those inthe above first embodiment are identified by the same referencenumerals, and the detailed description will be skipped or simplified.

An imaging device according to the second embodiment is different fromthe imaging device 1 described in the above first embodiment in that theimaging device according to the second embodiment generates shootingcondition information in consideration of a place where a user of theimaging device is in upon shooting as well as a result of an analysis ofvoice data.

Configuration of Imaging Device

FIG. 6 is a block diagram illustrating a configuration of an imagingdevice 1A according to the second embodiment of the present invention.

As illustrated in FIG. 6, the imaging device 1A according to the secondembodiment additionally includes a position acquiring unit 18 withrespect to the imaging device 1 (FIG. 1) described in the firstembodiment.

The position acquiring unit 18 receives satellite orbit informationtransmitted from plural GPS satellites, which form a GPS (GlobalPositioning System) serving as a measuring unit for measuring a positionof an object on the ground, and acquires position information(information related to a longitude and a latitude) of the imagingdevice 1A based on the received orbit information. The positionacquiring unit 18 then outputs the position information of the imagingdevice 1A to a control unit 17A.

A recording unit 16A according to the second embodiment is configured byadding a map information recording unit 164 to the recording unit 16(FIG. 1) described in the above first embodiment.

The map information recording unit 164 records map information includinga longitude and a latitude of each point.

A part of a function of a shooting condition information generating unit174A according to the second embodiment is changed according to theadditional position acquiring unit 18 and the map information recordingunit 164, compared to the shooting condition information generating unit174 described in the above first embodiment.

Operation of Imaging Device

FIG. 7 is a flowchart illustrating an operation of the imaging device1A.

As illustrated in FIG. 7, the operation of the imaging device 1Aaccording to the present second embodiment is different from theoperation (FIG. 3) of the imaging device 1 described in the above firstembodiment only in that step S111 is added, and step S109A (shootingcondition information generating step) is executed in place of stepS109.

Therefore, only steps S111 and S109A will be described below.

Step S111 is executed after step S108.

Specifically, the control unit 17A stores position information(information concerning the latitude and the longitude of the imagingdevice LA), which is acquired by the position acquiring unit 18 at thepoint at which the shooting operation is executed (step S104: Yes), intothe memory unit 15 (step S111).

Then, the shooting condition information generating unit 174A generatesshooting condition information based on the analysis result by the voicedata analyzing unit 173 in step S108, the related information recordedin the related information recording unit 163, the position informationof the imaging device 1A stored in the memory unit 15 in step S111, andthe map information recorded in the map information recording unit 164(step S109A).

Specifically, in step S109A, the shooting condition informationgenerating unit 174A generates the shooting condition informationincluding the place where the photographer is in upon shooting (stepS104: Yes) in addition to the shooting condition information generatedin step S109 described in the above first embodiment, the place beingdetermined from the position information of the imaging device 1A andthe map information recorded in the map information recording unit 164.

Specific Example of Image File

FIG. 8 is a view illustrating one example of an image corresponding toan image file generated by the operation of the imaging device 1A.

The image illustrated in FIG. 8 corresponds to the image file generatedby imaging under the shooting condition illustrated in FIG. 4.

Under the shooting condition illustrated in FIG. 4, for example, theshooting condition information generating unit 174A reads the “familyphoto”, which is associated with the specific voice “Mom” uttered fromthe daughter Sd, from the related information pieces (FIG. 2) recordedin the related information recording unit 163, as in the above firstembodiment. The shooting condition information generating unit 174A alsorecognizes that the place where the photographer (mother Sm) is in is a“zoo” based on the position information of the imaging device 1A storedin the memory unit 15 in step S111 and the map information recorded inthe map information recording unit 164. The shooting conditioninformation generating unit 174A generates the read “family photo” andthe recognized “zoo” as the shooting condition information (step S109A).Thereafter, the recording controller 175 generates an image file inwhich the “family photo” and the “zoo” that are the shooting conditioninformation are associated with the recording image data stored in thememory unit 15 in step S105, and records this image file in the imagedata recording unit 162 (step S110).

As described above, after the image file is recorded, the imaging device1A reads the image file with which the shooting condition informationmatching or similar to the inputted keyword (e.g., “family photo” or“zoo”), out of the image files recorded in the image data recording unit162, according to the playback operation to the input unit 13 by theuser of the imaging device 1A, and displays an image 1200 (FIG. 8)corresponding to this image file.

Specifically, the image 1200 is an image in which a character image T2of “family photo (zoo)” that is the shooting condition information issuperimposed on an image W1 corresponding to the recording image datagenerated according to the shooting operation to the input unit 13, asillustrated in FIG. 8.

The second embodiment described above brings the following effects inaddition to the effects of the above first embodiment.

The imaging device 1A according to the second embodiment generatesshooting condition information in consideration of the place where theuser of the imaging device 1A is in upon shooting, as well as theanalysis result of voice data.

Accordingly, the imaging device 1A can generate shooting conditioninformation having a high likelihood of matching the actual shootingcondition, and associate this shooting condition information withrecording image data.

Modification of Second Embodiment

The imaging device 1A according to the second embodiment generatesshooting condition information based on the analysis result of voicedata and the place where the user of the imaging device 1A is in uponshooting. However, the imaging device 1A is not limited thereto. Forexample, the imaging device 1A may generate shooting conditioninformation in consideration of a current date (date that is generatedby the imaging unit 11 and at which recording image data is stored inthe memory unit 15) provided from a real time clock (not illustrated).

For example, when the place where the mother Sm (the user of the imagingdevice) and the daughter Sd are in is not a zoo but an “elementaryschool”, and the current date is “X:XX, March O” under the shootingcondition illustrated in FIG. 4, the shooting condition informationgenerating unit generates the shooting condition information as“graduation ceremony”. For example, when the place where the mother Smand the daughter Sd are in is not a zoo but a “shrine”, and the currentdate is “X:XX, November O” under the shooting condition illustrated inFIG. 4, the shooting condition information generating unit generates theshooting condition information as “Shichi-go-san (the seven-five-threefestival)”.

By considering also the current date as described above, the imagingdevice can generate shooting condition information having morelikelihood of matching the actual shooting condition, and associate thisshooting condition information with recording image data.

Third Embodiment

Next, a third embodiment of the present invention will be described.

In the description below, the configurations and steps same as those inthe above first embodiment are identified by the same referencenumerals, and the detailed description will be skipped or simplified.

An imaging device according to the third embodiment is different fromthe imaging device 1 described in the above first embodiment in that theimaging device according to the third embodiment collects voices fromboth a photographer and a subject, and determines an age or a sex of theperson uttering a voice by analyzing voice data based on the respectivecollected voices. The analyzing function of voice data of the imagingdevice according to the third embodiment is different from the analyzingfunction of voice data described in the first embodiment, so thatrelated information is also different from the related informationdescribed in the first embodiment.

Configuration of Imaging Device

FIG. 9 is a block diagram illustrating an imaging device 1B according tothe third embodiment of the present invention.

As illustrated in FIG. 9, the imaging device 1B according to the thirdembodiment employs a voice data generating unit 14B in place of thevoice data generating unit 14 in the imaging device 1 (FIG. 1) describedin the first embodiment.

As illustrated in FIG. 9, the voice data generating unit 14B includes afirst voice data generating unit 14B1 and a second voice data generatingunit 14B2.

The first voice data generating unit 14B1 includes a first voice inputunit 141B1 and a first voice processing unit 142B1, each having thesimilar configuration and function to the voice input unit 141 and thevoice processing unit 142 described in the first embodiment.

The second voice data generating unit 14B2 includes a second voice inputunit 141B2 and a second voice processing unit 142B2, each having thesimilar configuration and function to the voice input unit 141 and thevoice processing unit 142 described in the first embodiment.

The arrangement positions of the first and second voice data generatingunits 14B1 and 14B2 are different from the arrangement positions of thefirst and second voice input units 141B1 and 141B2.

The arrangement positions of the first and second voice input units141B1 and 141B2 will be described below.

FIG. 10 is a perspective view illustrating the imaging device 1B viewedfrom the side (back side) facing a photographer. FIG. 11 is aperspective view illustrating the imaging device 1B viewed from the side(front side) facing a subject.

As illustrated in FIG. 10, a first microphone hole 143 that extendsthrough an inside and outside of an outer casing for collecting a voicefrom a photographer is formed on the side (back side) facing thephotographer of the imaging device 1B. The first voice input unit 141B1is disposed at the position opposite to the first microphone hole 143 atthe inside of the outer casing for collecting mainly a voice utteredfrom the photographer via the first microphone hole 143.

Similarly, as illustrated in FIG. 11, a second microphone hole 144 thatextends through an inside and outside of an outer casing for collectinga voice from a subject is formed on the side (front side) facing thesubject of the imaging device 1B. The second voice input unit 141B2 isdisposed at the position opposite to the second microphone hole 144 atthe inside of the outer casing for collecting mainly a voice utteredfrom the subject via the second microphone hole 144.

For the sake of convenience of description, the voice data generated bythe first voice data generating unit 14B1 is described as “first voicedata”, and the voice data generated by the second data generating unit14B2 is described as “second voice data”.

As illustrated in FIG. 9, the imaging device 1B according to the thirdembodiment employs a voice data analyzing unit 173B in place of thevoice data analyzing unit 173 in the imaging device 1 (FIG. 1) describedin the first embodiment.

The voice data analyzing unit 173B analyzes the first and second voicedata pieces generated respectively by the first and second voice datagenerating units 14B1 and 14B2.

The voice data analyzing unit 173B has “a function of determiningwhether a person uttering a voice is a photographer or a subject” and “afunction of determining an age or a sex of a person uttering a voice(age/sex analysis)” as well as the function (the function of determiningwhether a specific voice is included or not in first and second voicedata pieces) similar to the voice data analyzing unit 173 described inthe above first embodiment.

The “function of determining whether a person uttering a voice is aphotographer or a subject” by the voice data analyzing unit 173B is asstated below.

Specifically, before conducting the age/sex analysis of a personuttering a voice, the voice data analyzing unit 173B compares first andsecond voice data pieces. When a portion with the same wavelength of avoice is included in first and second voice data pieces at the sametiming, the voice data analyzing unit 173B defines the portion with ahigher level of an electric signal out of the portions in the first andsecond voice data pieces as a subject of the age/sex analysis.

For example, when a photographer utters a word, the first voice inputunit 141B1 disposed at the side of the photographer collects this word(voice), and the second voice input unit 141B2 disposed at the side of asubject may also collect this word (voice). Specifically, first andsecond voice data pieces generated by the first and second voice datagenerating units 14B1 and 14B2 respectively include a portioncorresponding to this word (voice). The portion included in the firstand second voice data pieces has the same timing and has the same voicewaveform. However, the level of the electric signal in the first voicedata is higher than that in the second voice data, since the first voiceinput unit 141B1 is located closer to the photographer than the secondvoice input unit 141B2. Therefore, the voice data analyzing unit 173Bdoes not define the portion in the second voice data as the subject ofthe age/sex analysis, but defines the portion in the first voice data asthe subject of the age/sex analysis.

Specifically, the voice data analyzing unit 173B compares the first andsecond voice data pieces as described above to select the subject of theage/sex analysis, in order to determine whether the person uttering avoice is a photographer or a subject.

For example, methods (1) and (2) can be described as examples of themethod of the “age/sex analysis” by the voice data analyzing unit 173B.

(1) The voice data analyzing unit 173B extracts formant from voice data(first and second voice data pieces). The voice data analyzing unit 173Bthen determines an age group or a sex of a person uttering a voice basedon the extracted formant (see Japanese Unexamined Patent Publication No.2007-304619, for example).(2) The voice data analyzing unit 173B compares a sex model and anage-group model generated beforehand and a voice based on voice data(first and second voice data pieces) to calculate similarity to eachmodel, and selects a category (sex, age group) with the highestsimilarity (see Japanese Unexamined Patent Publication No. 2006-285115,for example).

Examples of usable methods for determining an age group include a methodof employing Gaussian mixture model (GMM) classifier with Mel-FrequencyCepstrum Coefficient (MFCC) being used as a characteristic amount, amethod of using a support vector machine (SVM) with MFCC being used as acharacteristic amount, a method of using a SVM classifier by usingspeaker-adapted GMM as a characteristic amount, a method of using a SVMclassifier by using maximum likelihood linear regression (MLLR)transformation matrix as a characteristic amount, and a method of usingGMM adapted to maximum a posteriori (MAP) as a characteristic amount, inaddition to the methods (1) and (2).

As illustrated in FIG. 9, a recording unit 16B in the imaging device 1Baccording to the third embodiment employs a related informationrecording unit 163B that records related information different from therelated information recorded in the related information recording unit163, instead of the related information recording unit 163 described inthe first embodiment.

FIG. 12 is a diagram for describing related information recorded in therelated information recording unit 163B. FIGS. 13A and 13B are diagramsillustrating one example of related information recorded in the relatedinformation recording unit 163B.

The related information recording unit 163B records related informationin which shooting condition information is associated with a specificvoice (extracted keyword) and an age group and a sex of a personuttering a voice.

For example, age groups and sexes of a subject and a photographer, whichare persons uttering a voice, are classified into patterns A to H asillustrated in FIG. 12.

Specifically, the pattern A indicates that a subject who is a “child(under age)” utters a voice, and a photographer who is a “child” uttersa voice. The pattern B indicates that a subject who is a “child” uttersa voice, and a photographer who is a “female (adult (of age))” utters avoice. The pattern C indicates that a subject who is a “child” utters avoice, and a photographer who is a “male (adult)” utters a voice. Thepattern D indicates that a subject who is a “child” utters a voice, anda photographer utters no voice (no candidate). The pattern E indicatesthat a subject who is a “female (adult)” or a “male (adult)” utters avoice, or a subject utters no voice (no candidate), and a photographerwho is a “child” utters a voice. The pattern F indicates that a subjectwho is a “female (adult)” or a “male (adult)” utters a voice, and aphotographer who is a “female (adult)” or a “male (adult)” utters avoice. The pattern G indicates that a subject utters no voice (nocandidate), and a photographer who is a “female (adult)” or a “male(adult)” utters a voice. The pattern G indicates that a subject who is a“female (adult)” or a “male (adult)” utters a voice, and a photographerutters no voice (no candidate). The pattern H indicates that neither asubject nor a photographer utters a voice (no candidate).

As illustrated in FIGS. 13A and 13B, in the related informationaccording to the third embodiment, the shooting condition informationassociated with a specific voice is different, when the patterndescribed above is different even if the specific voice is “Dad” or“Mom”.

Specifically, in the pattern A, a “family photo when a child is young(photo of a child shot by another child)” is associated as the shootingcondition information. In the pattern B, a “family photo when a child isyoung (photo shot by mother)” is associated as the shooting conditioninformation. In the pattern C, a “family photo when a child is young(photo shot by father)” is associated as the shooting conditioninformation. In the pattern D, a “photo when a child is young” isassociated as the shooting condition information. In the pattern E, a“family photo when children are young (photo of a child shot by anotherchild)” is associated as the shooting condition information. In thepattern F, a “family photo after a child grows up (photo shot by aparent or a child)” is associated as the shooting condition information.In the pattern G, a “family photo (shot by a parent or a child)” isassociated as the shooting condition information. In the pattern H,shooting condition information indicating that the shooting condition isunclear is associated.

As illustrated in FIGS. 13A and 13B, in the related informationaccording to the third embodiment, the shooting condition informationassociated with a specific voice is different, when the patterndescribed above is different even if the specific voice is “Teacher”.

Specifically, in the pattern A, a “school event (photo of a student shotby another student)” is associated as the shooting conditioninformation. In the pattern B, a “school event (photo shot by a femaleteacher)” is associated as the shooting condition information. In thepattern C, a “school event (photo shot by a male teacher)” is associatedas the shooting condition information. In the pattern D, a “school event(photo shot by a teacher)” is associated as the shooting conditioninformation. In the pattern E, a “school event (photo of a teacher shotby a student)” is associated as the shooting condition information. Inthe pattern F, a “school event (photo shot by a teacher, or a photo of ateacher)” is associated as the shooting condition information. In thepattern G, a “school event” is associated as the shooting conditioninformation. In the pattern H, shooting condition information indicatingthat the shooting condition is unclear is associated.

Specific Example of Image File

The operation of the imaging device 1B according to the third embodimentis almost similar to the operation (FIG. 3) of the imaging device 1described in the first embodiment, so that the detailed description willnot be repeated. A specific example of an image file generated by theoperation of the imaging device 1B according to the third embodimentwill be described below.

FIG. 14 is a view illustrating one example of an image corresponding toan image file generated by the operation of the imaging device 1B.

The image in FIG. 14 corresponds to an image file generated by the imagecapture under the shooting condition illustrated in FIG. 4.

Under the shooting condition illustrated in FIG. 4, for example, thevoice data analyzing unit 173B analyzes first and second voice datapieces, which are generated by the first and second voice datagenerating units 14B1 and 14B2 and stored in the memory unit 15 duringsteps S102 to S107, as described below (step S108).

The voice data analyzing unit 173B compares first and second voice datapieces including the word “Look this way” uttered from the mother Sm,and recognizes that the person uttering this word is a photographer froma difference in levels of an electric signal of a voice part in the word“Look this way”. The voice data analyzing unit 173B also analyzes firstand second voice data pieces including the word “Mom, elephant” utteredfrom the daughter Sd, and recognizes that the person uttering this wordis a subject from a difference in levels of an electric signal of avoice part in the word “Mom, elephant”.

The voice data analyzing unit 173B then performs the age/sex analysis tothe first voice data including the word “Look this way” uttered from thephotographer (mother Sm) to recognize that the photographer is a “female(adult)”. The voice data analyzing unit 173B also performs the age/sexanalysis to the second voice data including the word “Mom, elephant”uttered from the subject (daughter Sd) to recognize that the subject isa “child”. Specifically, the voice data analyzing unit 173B recognizesthat the pattern forming the related information is the pattern B (FIG.12) as a result of the age/sex analysis described above.

In addition, the voice data analyzing unit 173B recognizes that thefirst and second voice data pieces include the specific voice “Mom”forming the related information recorded in the related informationrecording unit 163B, as in the above first embodiment.

A shooting condition information generating unit 174B according to thethird embodiment reads the “family photo when a child is young (photoshot by mother)” associated with the specific voice “Mom” and thepattern B from the related information (FIGS. 13A and 13B) recorded inthe related information recording unit 163B (step S109). Thereafter, therecording controller 175 generates an image file in which the “familyphoto when a child is young (photo shot by mother)” is associated withthe recording image data stored in the memory unit 15 in step S105, andrecords this image file in the image data recording unit 162 (stepS110).

After the image file is recorded as described above, the imaging device1B reads an image file with which shooting condition informationmatching or similar to an inputted keyword (e.g., “family photo when achild is young” or “photo shot by mother”) is associated, out of theimage files recorded in the image data recording unit 162, according tothe playback operation to the input unit 13 by the user of the imagingdevice 1B, and displays an image 1300 (FIG. 14) corresponding to thisimage file.

Specifically, as illustrated in FIG. 14, the image 1300 is an image inwhich a character image T3 of “family photo when a child is young (photoshot by mother)” that is the shooting condition information issuperimposed on an image W1 corresponding to the recording image datagenerated according to the shooting operation to the input unit 13.

The third embodiment described above brings the following effects inaddition to the effects similar to those of the first embodiment.

The imaging device 1B according to the third embodiment collects voicesfrom both a photographer and a subject, and analyzes voice data based oneach of the respective collected voices. The imaging device 1Bdetermines an age group or a sex of a person uttering the voice byanalyzing the voice data.

With this, the imaging device 1B can determine whether a person utteringa voice is a photographer or a subject. Further, the imaging device 1Bcan generate new shooting condition information (photo shot by OO, XXphoto, etc.) for specifying a photographer or a subject, and associatethis shooting condition information with recording image data.

Modification of Third Embodiment

When a photographer is determined to be a “child” (the patterns A and Ein FIG. 12) as a result of the analysis by the voice data analyzing unit173B in the third embodiment, the imaging device 1B may be configuredsuch that A control unit 17B executes an assist function for assistingthe shooting by the photographer.

Examples of the assist function include an automatic zooming function tochange a zooming position of an optical system, which composes theimaging unit 11, in order to allow the whole subject such as a person tofall within a viewing angle range, and a layout assist function todisplay a layout assist guide.

According to the execution of the assist function in the above case,even a child who is unfamiliar to the operation of the imaging devicecan shoot a satisfactory photograph, whereby usability can be enhanced.

In the third embodiment described above, the configurations described inthe above second embodiment may be employed for the imaging device 1B togenerate shooting condition information in consideration of a placewhere the user of the imaging device 1B is in upon shooting and thecurrent date.

The above third embodiment illustrates related information consideringmajority human relationships (FIGS. 12, 13A, and 13B). However, therelated information may be changed, as necessary, to meet needs of auser, considering a recent situation in which same-sex couples arepresent, although such situation has a low percentage. A keyworddetermined from the voice may be based on a user's language, a dialectin a district, or custom in a district. For example, it is obvious thata user image is specified by a user's action pattern or posting in SNS(Social Networking Service), and optimized by utilizing analysis data ofbig data. It is considered that a person name posted in SNS is replacedby a keyword, and its relation is analyzed to be used as reference forpattern classification. In addition, when a user keeps a certain animalas a pet, a pattern in which this pet is around the user if the imagingdevice catches its voice can be formed, and attaching the meaning of“photo when the pet cries” to the image (“photo when the pet cries” isassociated with the recording image data as shooting conditioninformation) is helpful for an image search or image viewing. Even ifthe pet is not in the photo, such information becomes significantinformation for viewing the image. The present embodiment also includesan application in which pet's feeling is associated with a photo from avoice of a spoiled dog or a spoiled cat. Recently, a wearable camera hasbeen developed. To create meaning to a captured image by effectivelyutilizing a voice that can be detected wherever the wearable camera isset can be expected to provide a significant effect to specify a personor a pet with a wearable camera, since image data is difficult to beacquired in this situation.

It is obvious that the above third embodiment can include an applicationin which intonation or rhythm is determined from a word or wordsextracted from voice data to determine even a feeling with the database,for example, and feeling information concerning the determined feelingis included in the shooting condition information.

The determination of even a feeling as described above further increasesan application range of the present invention, with the result that afeeling of a photographer or a subject as well as an objective conditionupon shooting can be reproduced. A viewer of an image not only simplyunderstands the image but also feels empathy for the image withemotional involvement. A situation in which a viewer searches aphotograph for which he/she can feel empathy with emotional involvement,and he/she can relive the feeling upon shooting is a significant featureof photographic culture. In addition, a voice is not recorded as it is,but is recorded as converted into a text implying the voice, wherebysearch performance is dramatically enhanced. Accordingly, a feeling thatis glorified but not vivid can be evoked, and restriction in playbacktime is not imposed.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described.

In the description below, the configurations and steps same as those inthe above first embodiment are identified by the same referencenumerals, and the detailed description will be skipped or simplified.

The imaging device according to the fourth embodiment is different fromthe imaging device 1 described in the first embodiment in that theimaging device according to the fourth embodiment generates shootingcondition information based on an analysis result of recording imagedata in addition to an analysis result of voice data. The function ofanalyzing voice data in the imaging device according to the fourthembodiment is similar to the “function of determining an age group or asex of a person uttering a voice” described in the above thirdembodiment.

Configuration of Imaging Device

FIG. 15 is a block diagram illustrating a configuration of an imagingdevice 1C according to the fourth embodiment.

As illustrated in FIG. 15, a control unit 17C composing the imagingdevice 1C according to the fourth embodiment is formed by additionallyincluding an image data analyzing unit 176 in the control unit 17(FIG. 1) described in the first embodiment.

The image data analyzing unit 176 analyzes recording image data storedin the memory unit 15.

Specifically, the image data analyzing unit 176 performs a facerecognition process to recording image data, and extracts featureinformation (shape feature information concerning a size or shape of anoutline of a face, eyes, nose, and mouth, and information concerningcolor or brightness of face skin, i.e., contrast feature informationconcerning “wrinkles” or “dullness”) in a region of a detected face. Theimage data analyzing unit 176 compares the extracted feature informationand age-group models or sex models recorded beforehand in a recordingunit 16C, and determines an age group or a sex of a person included inthe recording image data from similarity to each model. The image dataanalyzing unit 176 may extract feature information in a neck region aswell as the feature information of a face region, and determine an agegroup or a sex of a person based on the feature information pieces.

As illustrated in FIG. 15, the imaging device 1C according to the fourthembodiment employs a voice data analyzing unit 173C, instead of thevoice data analyzing unit 173 in the imaging device 1 (FIG. 1) describedin the above first embodiment.

The voice data analyzing unit 173C analyzes voice data stored in thememory unit 15 to perform the age/sex analysis described in the abovethird embodiment.

As illustrated in FIG. 15, the imaging device 1C according to the fourthembodiment employs a related information recording unit 163C, whichrecords related information different from the related informationrecorded in the related information recording unit 163, instead of therelated information recording unit 163 described in the above firstembodiment.

FIG. 16 is a diagram for describing related information recorded in therelated information recording unit 163C. FIGS. 17A and 17B are diagramsillustrating one example of related information.

The related information recording unit 163C records related informationin which shooting condition information is associated with an age group,sex, and number of persons uttering a voice, and an age group, sex, andnumber of persons who are subjects.

For example, as illustrated in FIG. 16, age groups, sexes, and number ofpersons uttering a voice and persons who are subjects are classifiedinto patterns I to S.

Specifically, the pattern I indicates that a person uttering a voice isone “child”, and a person who is a subject is a “child”. The pattern Jindicates that a person uttering a voice is a “child”, and persons whoare subjects are “one male and one female (adult)”. The pattern Kindicates that a person uttering a voice is a “child”, and personsuttering a voice are “two same sex persons (adult)”. The pattern Lindicates that a person uttering a voice is a “child”, and a person whois a subject is not present (no candidate). The pattern M indicates thatpersons uttering a voice are “one male and one female (adult)”, and aperson who is a subject is a “child”, or “two same sex persons (adult)”.The pattern M also indicates that persons uttering a voice are “two samesex persons (adult)” and a person who is a subject is a “child” or “onemale and one female (adult)”. The pattern M also indicates that personsuttering a voice are “one male and one female (adult)” or “two same sexpersons (adult)”, and a person who is a subject is not present (nocandidate). The pattern N indicates that persons uttering a voice are“one male and one female (adult)”, and persons who are subjects are “onemale and one female (adult)”. The pattern O indicates that personsuttering a voice are “two same sex persons (adult)”, and persons who aresubjects are “two same sex persons (adult)”. The pattern P indicatesthat a person uttering a voice is not present (no candidate), and aperson who is a subject is a “child”. The pattern Q indicates that aperson uttering a voice is not present (no candidate), and persons whoare subjects are “one male and one female (adult)”. The pattern Rindicates that a person uttering a voice is not present (no candidate),and persons who are subjects are “two same sex persons (adult)”. Thepattern S indicates that a person uttering a voice is not present (nocandidate), and a person who is a subject is not present (no candidate).

As illustrated in FIGS. 17A and 17B, the related information accordingto the fourth embodiment is formed such that different shootingcondition information is associated with each of the patterns I to S.

Specifically, a “photo of a child (talking)” is associated with thepattern I as the shooting condition information. A “photo of one maleand one female (shot by a child)” is associated with the pattern J asthe shooting condition information. A “photo of two same sex persons(shot by a child)” is associated with the pattern K as the shootingcondition information. A “photo shot by a child” is associated with thepattern L as the shooting condition information. A “photo shot by twoclose friends” is associated with the pattern M as the shootingcondition information. A “heart mark” is associated with the pattern Nas the shooting condition information. An “in-conversation mark” isassociated with the pattern O as the shooting condition information. A“photo of a child” is associated with the pattern P as the shootingcondition information. A “photo of one male and one female” isassociated with the pattern Q as the shooting condition information. A“photo of two same sex persons” is associated with the pattern R as theshooting condition information. Shooting condition informationindicating that the shooting condition is unclear is associated with thepattern S.

In the fourth embodiment, the shooting condition information isdifferent between the case where the age group, sex, and number of thesubject are the same as the age group, sex, and number of the personsuttering a voice (patterns I, N, and O in FIG. 16) and the case wherethey are different (patterns J to M, and P to S in FIG. 16).

Operation of Imaging Device

FIG. 18 is a flowchart illustrating the operation of the imaging device1C.

As illustrated in FIG. 18, the operation of the imaging device 1Caccording to the fourth embodiment is different from the operation (FIG.3) of the imaging device 1 described in the above first embodiment inthat step S108C (voice data analyzing step) and S109C (shootingcondition generating step) are added in place of steps S108 and S109,and step S112 is added.

Therefore, only steps S108C, S112, and S109C will be described below.

In step S108C, the voice data analyzing unit 173C analyzes voice data,which is generated by the voice data generating unit 14 and stored inthe memory unit 15 in steps S102 to S107, thereby conducting an age/sexanalysis (including an analysis of a number of persons uttering avoice), as in the above third embodiment.

Then, in step S112, the image data analyzing unit 176 reads therecording image data stored in the memory unit 15 in step S105, andanalyzes the recording image data to determine an age group, sex, andnumber of persons included in the recording image data.

In step S109C, A shooting condition information generating unit 174Caccording to the fourth embodiment reads (generates) shooting conditioninformation associated with the analysis result by the voice dataanalyzing unit 173C in step S108C and the analysis result by the imagedata analyzing unit 176 in step S112 from the related information piecesrecorded in the related information recording unit 163C.

Specific Example of Image File

FIGS. 19A to 19C are views illustrating one example of an imagecorresponding to an image file generated by the operation of the imagingdevice 1C.

An image 1400 illustrated in FIG. 19A is an image corresponding to animage file generated by imaging under the shooting condition describedbelow.

This shooting condition is such that a photographer (mother) capturesher daughter as a subject with an elephant in the background in a zoo,like the shooting condition illustrated in FIG. 4.

In this case, the mother who is a photographer utters no word upon theshooting operation with the imaging device 1C. On the other hand, herdaughter who is a subject utters some word to her mother during apredetermined period (steps S102 to S107) including the point of theshooting operation by her mother with the imaging device 1C.

In such shooting condition, the voice data analyzing unit 173C performs(step S108C) an age/sex analysis (including an analysis of a number ofpersons uttering a voice) for the voice data generated in steps S102 toS107 to recognize that the person uttering a voice is one “child”.

The image data analyzing unit 176 analyzes the recording image datastored in the memory unit 15 in step S105 to recognize that the person,who is a subject, is one “child” (step S112).

Specifically, the pattern forming the related information is the patternI (FIG. 16) from the analysis results of the voice data analyzing unit173C and the image data analyzing unit 176.

The shooting condition information generating unit 174C reads the “photoof a child (talking)” associated with the pattern I from the relatedinformation pieces (FIGS. 17A and 17B) recorded in the relatedinformation recording unit 163C (step S109C). Thereafter, the recordingcontroller 175 generates an image file in which the “photo of a child(talking)” that is the shooting condition information is associated withthe recording image data stored in the memory unit 15 in step S105, andrecords this image file onto the image data recording unit 162 (stepS110).

After the image file is recorded as described above, the imaging device1C reads the image file with which the shooting condition informationmatching or similar to the inputted keyword (e.g., “photo of a child” or“talking”), out of the image files recorded in the image data recordingunit 162, according to the playback operation to the input unit 13 bythe user of the imaging device 1C, and displays the image 1400 (FIG.19A) corresponding to this image file.

Specifically, the image 1400 is an image in which a character image T4of “a photo of a child (talking)” that is the shooting conditioninformation is superimposed on an image W1 corresponding to therecording image data generated according to the shooting operation tothe input unit 13, as illustrated in FIG. 19A.

An image 1500 illustrated in FIG. 19B corresponds to an image filegenerated by imaging under the shooting condition described below.

This shooting condition is such that a photographer captures one maleand one female (they are adults) as subjects with an elephant in thebackground in a zoo.

In this case, the photographer utters no word upon the shootingoperation with the imaging device 1C. On the other hand, the male andthe female who are subjects have a talk during a predetermined period(steps S102 to S107) including the point of the shooting operation bythe photographer with the imaging device 1C.

In such shooting condition, the voice data analyzing unit 173C performs(step S108C) an age/sex analysis (including an analysis of a number ofpersons uttering a voice) for the voice data generated in steps S102 toS107 to recognize that the persons uttering a voice are “one male andone female (adults)”.

The image data analyzing unit 176 analyzes the recording image datastored in the memory unit 15 in step S105 to recognize that the persons,who are subjects, are “one male and one female (adult)” (step S112).

Specifically, the pattern forming the related information is the patternN (FIG. 16) from the analysis results of the voice data analyzing unit173C and the image data analyzing unit 176.

The shooting condition information generating unit 174C reads the “heartmark” associated with the pattern N from the related information pieces(FIG. 17) recorded in the related information recording unit 163C (stepS109C). Thereafter, the recording controller 175 generates an image filein which the “heart mark” that is the shooting condition information isassociated with the recording image data stored in the memory unit 15 instep S105, and records this image file onto the image data recordingunit 162 (step S110).

After the image file is recorded as described above, the imaging device1C reads the image file with which the shooting condition informationmatching or similar to the inputted keyword (e.g., “heart mark” or“heart”), out of the image files recorded in the image data recordingunit 162, according to the playback operation to the input unit 13 bythe user of the imaging device 1C, and displays the image 1500 (FIG.19B) corresponding to this image file.

Specifically, the image 1500 is an image in which a graphic image T5 ofthe “heart mark” that is the shooting condition information issuperimposed between two persons on an image W2 corresponding to therecording image data generated according to the shooting operation tothe input unit 13, as illustrated in FIG. 19B.

An image 1600 illustrated in FIG. 19C corresponds to an image filegenerated by imaging under the shooting condition described below.

This shooting condition is such that a photographer captures two females(they are adults) as subjects with an elephant in the background in azoo.

In this case, the photographer utters no word upon the shootingoperation with the imaging device 1C. On the other hand, the two femaleswho are subjects have a talk during a predetermined period (steps S102to S107) including the point of the shooting operation by thephotographer with the imaging device 1C.

In such shooting condition, the voice data analyzing unit 173C performsan age/sex analysis (including an analysis of a number of personsuttering a voice) for the voice data generated in steps S102 to S107 torecognize that the persons uttering a voice are “two females (adults)”(step S108C).

The image data analyzing unit 176 analyzes the recording image datastored in the memory unit 15 in step S105 to recognize that the persons,who are subjects, are “two females (adult)” (step S112).

Specifically, the pattern forming the related information is the patternO (FIG. 16) from the analysis results of the voice data analyzing unit173C and the image data analyzing unit 176.

The shooting condition information generating unit 174C reads the“in-conversation mark” associated with the pattern O from the relatedinformation pieces (FIGS. 17A and 17B) recorded in the relatedinformation recording unit 163C (step S109C). Thereafter, the recordingcontroller 175 generates an image file in which the “in-conversationmark” that is the shooting condition information is associated with therecording image data stored in the memory unit 15 in step S105, andrecords this image file onto the image data recording unit 162 (stepS110).

After the image file is recorded as described above, the imaging device1C reads the image file with which the shooting condition informationmatching or similar to the inputted keyword (e.g., “in conversation”),out of the image files recorded in the image data recording unit 162,according to the playback operation to the input unit 13 by the user ofthe imaging device 1C, and displays the image 1600 (FIG. 19C)corresponding to this image file.

Specifically, the image 1600 is an image in which a graphic image T6 ofthe “in-conversation mark” that is the shooting condition information issuperimposed between two persons on an image W3 corresponding to therecording image data generated according to the shooting operation tothe input unit 13, as illustrated in FIG. 19C.

The graphic image T6 is formed such that a character of “inconversation” is written in a double-headed arrow pointing each of twopersons in the image W3.

The present fourth embodiment described above brings the followingeffects as well as the effects similar to the above first embodiment.

The imaging device 1C according to the fourth embodiment generatesshooting condition information based on the analysis result of voicedata and the analysis result of recording image data. The imaging device1C determines an age group or a sex of a person uttering a voice byanalyzing voice data, and determines an age group or a sex of a person,who is a subject, by analyzing recording image data.

With this, the imaging device 1C can determine whether a person utteringa voice is a photographer or a subject. Further, the imaging device 1Ccan generate new shooting condition information (photo shot by OO, XXphoto, etc.) for specifying a photographer or a subject, and associatethis shooting condition information with recording image data.

In the fourth embodiment, the shooting condition information isdifferent between the case where the age group, sex, and number of thesubject are the same as the age group, sex, and number of the personsuttering a voice (patterns I, N, and O in FIG. 16) and the case wherethey are different (patterns J to M, and P to S in FIG. 16).

Specifically, when the analysis result of voice data matches theanalysis result of recording image data, the imaging device 1Cdetermines that the person uttering a voice and a subject are the same,and can associate new shooting condition information (“talking”, “heartmark”, “in conversation”, etc.) describing the condition of the subjectwith the recording image data.

Modification of Fourth Embodiment

FIG. 20 is a diagram illustrating one example of related informationaccording to a modification of the fourth embodiment of the presentinvention.

Specifically, the related information in FIG. 20 is information inwhich, when subjects are two persons, and one of them is a “child”,different shooting condition information is associated depending on theage group and the sex of the other one.

In the above fourth embodiment, when the recording image data includesimages of plural persons, and one of them is a “child”, the shootingcondition information illustrated in FIG. 20 may be associated with therecording image data in steps S109C and S110, as well as the shootingcondition information described in the above fourth embodiment.

Specifically, when the persons who are the subjects are determined to betwo “children” as the analysis result (step S112) by the image dataanalyzing unit 176, the shooting condition information of “closefriends” is associated with the recording image data as well as theshooting condition information described in the above fourth embodimentin steps S109C and S110.

When the persons who are the subjects are determined to be one “child”and one “female (adult)” as the analysis result by the image dataanalyzing unit 176 (step S112), the photographer is likely to be afather. Therefore, the shooting condition information of “photo shot byfather” is associated with the recording image data as well as theshooting condition information described in the above fourth embodimentin steps S109C and S110.

When the persons who are the subjects are determined to be one “child”and one “male (adult)” as the analysis result by the image dataanalyzing unit 176 (step S112), the photographer is likely to be amother. Therefore, the shooting condition information of “photo shot bymother” is associated with the recording image data as well as theshooting condition information described in the above fourth embodimentin steps S109C and S110.

In the above fourth embodiment, when the person uttering a voice isdetermined to be a “child” as the analysis result by the voice dataanalyzing unit 173C, and the subject is determined not to include a“child” as the analysis result by the image data analyzing unit 176 (inFIG. 16, the patterns J to L), the control unit 17C may execute theassist function for assisting the shooting operation by thephotographer.

Specifically, the control unit 17C has a function as an assist functionexecuting unit according to the present invention.

Examples of the assist function include an automatic zooming function tochange a zooming position of an optical system, which composes theimaging unit 11, in order to allow the whole subject such as a person tofall within a viewing angle range, and a layout assist function todisplay a layout assist guide.

According to the execution of the assist function in the above case,even a child who is unfamiliar to the operation of the imaging devicecan shoot a satisfactory photo, whereby usability can be enhanced.

In the fourth embodiment described above, the configurations describedin the above second embodiment may be employed for the imaging device 1Cto generate shooting condition information in consideration of a placewhere the user of the imaging device 1C is in upon shooting and thecurrent date.

In the fourth embodiment described above, the configuration (theconfiguration in which the imaging device collects voices from both aphotographer and a subject, and determines whether the person uttering avoice is a photographer or a subject) described in the above thirdembodiment may be employed for the imaging device 1C.

The above fourth embodiment illustrates related information consideringmajority human relationships (FIGS. 16, 17A, and 17B). However, therelated information may be changed, as necessary, to meet needs of auser, considering a recent situation in which same-sex couples arepresent, although such situation has a low percentage. A keyworddetermined from the voice may be based on a user's language, a dialectin a district, or custom in a district. For example, it is obvious thata user image is specified by a user's action pattern or posting in SNS,and optimized by utilizing analysis data of big data. It is consideredthat a person name posted in SNS is replaced by a keyword, and itsrelation is analyzed to be used as reference for pattern classification.A voice includes a sweet voice, angry voice, gruff voice, and rushingvoice, and the condition indicated by the generated voice can bespecified according to the pattern of the voice (accent, tempo, or pitchof the keyword). Whether the persons are “close friends” or not, orwhether the “heart mark” should be attached or not can more surely bedetermined by utilizing such information. The present embodiment caneffectively be used for much more classifications. When a crying child(this is detected from his/her crying face) is present in an image, andan angry voice is determined, the imaging device can associate theshooting condition information (feeling information) of “being scolded”with the recording image data. On the contrary, a person with an angryface is in an image, and the imaging device catches crying, the imagingdevice can express this situation as “angry”. However, when a laughingvoice is detected, the imaging device can determine this situation as“playful”. In this way, the imaging device can associate more detailedshooting condition information (feeling information) with recordingimage data. The association of emotions with recording image data asshooting condition information as described above is helpful for animage search or image viewing. The emotions can be determined from anarrangement or angle of parts of a face composing a face image in animage, and also determined from the voice patterns described above or aspecific rhythm or a change in a pitch included in “laughing voice” or“crying voice”. As for the analysis of image data, a pose of a subjectas well as an expression of a face described above may be analyzed. Aperson may take a specific pose with a specific feeling. Therefore, whenit is determined that a person takes a specific pose, specific feelingcorresponding to this specific pose may be associated with recordingimage data as shooting condition information. In addition, when a userkeeps a certain animal as a pet, a pattern in which this pet is aroundthe user if the imaging device catches its voice can be formed, andattaching the meaning of “photo when the pet cries” to the image ishelpful for an image search or image viewing. Even if the pet is not inthe photo, such information becomes significant information for viewingthe image. The present embodiment also includes an application in whichpet's feeling is associated with a photo from a voice of a spoiled dogor a spoiled cat. Recently, a wearable camera has been developed. Tocreate meaning to a captured image by effectively utilizing a voice thatcan be detected wherever the wearable camera is set can be expected toprovide a significant effect to specify a person or a pet with awearable camera, since image data is difficult to be acquired in thissituation.

Fifth Embodiment

A fifth embodiment of the present invention will be described next.

In the description below, the configurations and steps same as those inthe above first embodiment are identified by the same referencenumerals, and the detailed description will be skipped or simplified.

The present fifth embodiment is configured to include the functions ofthe voice data analyzing unit 173 and the shooting condition informationgenerating unit 174 described in the first embodiment to an SNS serverthat is accessible via a network.

Configuration of Imaging Device

FIG. 21 is a block diagram illustrating a configuration of an imagingsystem 3 according to the fifth embodiment.

The imaging system 3 includes an imaging device 1D and an SNS server 2that bi-directionally communicate via a network N as illustrated in FIG.21.

As illustrated in FIG. 21, the imaging device 1D according to the fifthembodiment additionally includes a communication unit 19 with respect tothe imaging device 1 (FIG. 1) described in the above first embodiment.

The communication unit 19 performs radio communication with the SNSserver 2 via the network N according to a predetermined radiocommunication standard.

The communication unit 19 has a function as a transmission unitaccording to the present invention.

As illustrated in FIG. 21, a control unit 17D in the imaging device 1Daccording to the fifth embodiment is configured by eliminating the voicedata analyzing unit 173, the shooting condition information generatingunit 174, and the recording controller 175 from the control unit 17(FIG. 1) described in the first embodiment and adding a communicationcontroller 177 to the control unit 17.

The communication controller 177 controls the operation of thecommunication unit 19 to transmit image data and voice data to thedesignated SNS server 2 via the network N by referring to connectiondestination information (information (IP address, for example) necessaryfor performing radio communication with the SNS server 2 via the networkN) recorded in a recording unit 16D.

Specifically, the communication controller 177 transmits voice datagenerated and stored in the memory unit 15 in steps S102 to S107 andrecording image data stored in the memory unit 15 in step S105 to theSNS server 2 after step S107 described in the above first embodiment.

The communication controller 177 controls the operation of thecommunication unit 19 according to an access operation (including anoperation of inputting a keyword for searching an image) for an accessto the SNS server 2 to the input unit 13 by the user of the imagingdevice 1D, and transmits an access signal (including the inputtedkeyword) to the designated SNS server 2 via the network N by referringto the connection destination information recorded in the recording unit16D.

As illustrated in FIG. 21, the recording unit 16D in the imaging device1D according to the fifth embodiment is configured by eliminating therelated information recording unit 163 from the recording unit 16(FIG. 1) described in the above first embodiment.

As illustrated in FIG. 21, the SNS server 2 includes a communicationunit 21, a database 22, and a control unit 23.

The communication unit 21 performs radio communication with the imagingdevice 1D or a portable device (not illustrated) such as a cellularphone via the network N according to a predetermined radio communicationstandard.

The communication unit 21 has a function as a receiving unit accordingto the present invention.

The database 22 records an image file generated by the control unit 23,and records related information described in the above first embodiment.

Specifically, the database 22 has a function as an image data recordingunit according to the present invention.

As illustrated in FIG. 21, the control unit 23 includes a communicationcontroller 231, a voice data analyzing unit 232, a shooting conditioninformation generating unit 233, and a recording controller 234.

The communication controller 231 controls the operation of thecommunication unit 21 according to an access signal (including anoperation of inputting a keyword for searching an image) transmittedfrom the imaging device 1D or a portable device via the network N toestablish communication with the imaging device 1D or the portabledevice. The communication controller 231 also controls a display mannerof a designated page of a designated browser according to the accesssignal to display an image, which corresponds to an image file withwhich shooting condition information matching or similar to the keywordincluded in the access signal is associated, on a display unit of theimaging device 1D or the portable device.

The voice data analyzing unit 232 performs the analysis similar to thevoice data analyzing unit 173 described in the above first embodiment tothe voice data received from the imaging device 1D via the communicationunit 21.

The shooting condition information generating unit 233 generatesshooting condition information, as in the shooting condition informationgenerating unit 174 described in the above first embodiment, byreferring to the related information pieces recorded in the database 22based on the analysis result of the voice data analyzing unit 232.

The recording controller 234 generates the image file in which theshooting condition information generated by the shooting conditioninformation generating unit 233 is associated with the recording imagedata received from the imaging device 1D via the communication unit 21,and records this image file on the database 22.

The effects similar to the above first embodiment can be provided evenin the case where the SNS server 2 has the functions of the voice dataanalyzing unit 173 and the shooting condition information generatingunit 174, as in the fifth embodiment described above.

Modification of Fifth Embodiment

In the fifth embodiment described above, the configurations described inthe above second embodiment may be employed for the imaging device 1D.Specifically, information concerning a place where the user of theimaging device 1D is in upon shooting or the current date upon shootingmay be transmitted to the SNS server 2 in image data and voice data. Inthis case, the SNS server 2 generates shooting condition information inconsideration of a place where the user of the imaging device 1D is inupon shooting and the current date upon shooting.

In the fifth embodiment described above, the function of analyzing voicedata (age/sex analysis) or the function of analyzing image data, whichare described in the above third embodiment or fourth embodiment, may beadditionally applied to the SNS server 2. Specifically, the imagingdevice 1D may generate shooting condition information based on theanalysis result of the voice data or the analysis result of the imagedata.

Other Embodiments

While embodiments for embodying the present invention have beendescribed above, the present invention should not be limited by theabove first to fifth embodiments.

In the above first to fifth embodiments, shooting condition informationis associated with recording image data that is a still image. However,the present invention is not limited thereto. When a moving image iscaptured, shooting condition information may be associated with thegenerated moving-image file.

In the above first to fifth embodiments, when voice data includeshonorific “chan”, for example, an extracted keyword including thehonorific “chan” may be used as shooting condition information.

For example, under the shooting condition illustrated in FIG. 4, themother Sm says “OO-chan, look this way”, the word “OO-chan” is used asthe shooting condition information. The “chan” is used on the assumptionthat the language is Japanese. However, it is obvious that thecorresponding word can be changed according to the other languages ordialects based on a sales area, user setting, or profile information.

In the above first to fifth embodiments, image data may be analyzed, andfrom the analysis of the image data, whether a person is young, old,male, or female (age/sex analysis) may be determined from a face in theimage data, an expression of the face may be analyzed, or a pose of asubject may be analyzed. A feeling that can be determined from theanalysis of image data is effectively used, and not only a specific wordin voice data is detected but also intonation or rhythm is determinedfrom the word or words, whereby even a feeling can be determined fromthe database, for example. This can obviously increase an applicationrange of the present invention. Accordingly, a feeling of a photographeror a subject as well as an objective condition upon shooting can bereproduced, whereby a viewer of an image not only simply understands theimage but also feels empathy for the image with emotional involvement. Asituation in which a viewer searches a photograph for which he/she canfeel empathy with emotional involvement, and he/she can relive thefeeling upon shooting is significant needs and feature of photographicculture. For example, “pleasant memory”, “bitter memory”, or “sad days”can be more intuitive than the search using words of “when”, “who”, or“where”. When a camera becomes wearable or is used as lifelog, thesearch described above may be more appropriate and user friendly.

The tables illustrated in each embodiment can be rewritten, asnecessary, since words or expressions are changed depending ontransition of time or any areas. This may be achieved by referring tobig data from profile information of a user, or may be processed throughvarious networks or the Internet.

In the present invention, a voice is not recorded as it is, but isconverted into a text indicating this voice and then recorded. Withthis, search performance is dramatically enhanced, resulting in that afeeling that is glorified but not vivid can be evoked, and restrictionin playback time is not imposed.

The order of the processes is not limited to the order in the flowchartdescribed in the above first to fifth embodiments, and may be changedwithin a consistent range.

The algorithm of the processes described by using the flowchart in thepresent specification can be written as a program. Such program may berecorded in a recording unit in a computer, or may be recorded on acomputer-readable recording medium. The program may be recorded on arecording unit or a recording medium when a computer or the recordingmedium is shipped as a product, or it may be recorded on a recordingunit or a recording medium by being downloaded via a communicationnetwork.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An imaging device comprising: an imagerconfigured to capture an image of a subject and generate image data; avoice data generating processor configured to generate voice data basedon an input voice; a voice data analyzing processor configured toanalyze the voice data generated by the voice data generating processor;a shooting condition information generating processor configured togenerate shooting condition information related to a shooting condition,based on a result of an analysis by the voice data analyzing processor;an image data recorder configured to record therein the image datagenerated by the imager; and a controller configured to associate theimage data generated by the imager with the shooting conditioninformation generated by the shooting condition information generatingprocessor.
 2. The imaging device according to claim 1, wherein the voicedata analyzing processor is configured to analyze the voice datagenerated based on a voice from a photographer or a subject.
 3. Theimaging device according to claim 1, wherein the voice data analyzingprocessor is configured to analyze the voice data generated based on anambient sound around the imaging device upon shooting and a voice from aphotographer or a subject.
 4. The imaging device according to claim 1,wherein the voice data analyzing processor is configured to analyze thevoice data generated based on a conversation between a photographer anda subject.
 5. The imaging device according to claim 1, furthercomprising an image data analyzer configured to analyze the image datagenerated by the imager to determine an age group or sex of an image ofa person included in the image data, wherein the voice data analyzingprocessor is configured to analyze the voice data to determine an agegroup or a sex of a person uttering the voice, and the shootingcondition information generator is configured to generate differentshooting condition information depending on whether or not the age groupor the sex of the image of the person determined by the image dataanalyzer matches the age group or the sex of the person determined bythe voice analyzer.
 6. The imaging device according to claim 1, furthercomprising: an image data analyzer configured to analyze the image datagenerated by the imager to determine an age group of an image of aperson included in the image data; and an assist function executerconfigured to execute an assist function for assisting a shootingoperation by a photographer, wherein the voice data analyzing processoris configured to analyze the voice data to determine an age group of aperson uttering the voice, and the assist function executer isconfigured to execute the assist function, when the age group of theperson determined by the voice data analyzing processor is less than aspecified age, and the image data analyzer determines that an image of aperson with an age group equal to or more than the specified age is notincluded in the image data.
 7. The imaging device according to claim 1,further comprising an image data analyzer configured to analyze theimage data generated by the imager to determine a sex of an image of aperson included in the image data, wherein the shooting conditioninformation generator is configured to generate the shooting conditioninformation based on the result of the analysis by the voice dataanalyzing processor and the sex of the image of the person determined bythe image data analyzer.
 8. The imaging device according to claim 1,further comprising an image data analyzer configured to analyze theimage data generated by the imager to determine a sex of an image of aperson included in the image data, wherein the voice data analyzingprocessor is configured to analyze the voice data to determine a sex ofa person uttering the voice, and the shooting condition informationgenerator is configured to generate different shooting conditioninformation depending on whether or not the sex of the image of theperson determined by the image data analyzer matches the sex of theperson determined by the voice data analyzer.